Streaming of media has quickly become a very popular service which will likely continue to grow immensely such that the sales of physical products and downloads with media are likely to be more or less replaced by streaming services in due course. The services discussed in this disclosure may, without limitation, be related to streaming of audio and/or visual content such as music and video which may be delivered as on-demand services or as services related to social media.
When a service has been delivered by a service provider in a media stream by means of a communication network to a user, it is of interest for the service provider to know how the user has experienced the delivered service in terms of quality, e.g. to find out if the service or the communication network has any shortcomings that need to be eliminated or reduced in some way. This opinion of the user is commonly referred to as Quality of Experience, QoE, which is essentially the user's subjective opinion of the quality of a delivered service.
A traditional way to obtain a user's opinion about a delivered service is to explicitly ask the user to provide feedback about the service in a questionnaire or the like. For example, the service provider may send out or otherwise present an inquiry form, questionnaire, or opinion poll to the customer with various questions related to the quality of the delivered service. This procedure is however associated with various problems, as outlined below.
Firstly, it is often difficult to motivate a user to take the time and trouble to answer such questions and send a response back to the service provider and users are often notoriously reluctant to provide their opinions on such matters. Thus the response rate is typically too low to provide a representative result. One way to motivate the user is to reward him/her in some way when submitting a response, e.g. by giving some gift or discount, which means added costs for the service provider. Secondly, it may also happen that once the user answers such questions some time may have passed and he/she may not remember exactly how the service quality was perceived, and the response may thus be less than truthful.
Subjective test may also be conducted where a panel of users is asked to evaluate perceived quality of some streamed media. Typically the quality is given as a score on a scale from, e.g., 1, indicating “bad”, to 5, indicating “excellent”. The scores may then be averaged over the participating users to form a representative opinion score, which may also be referred to as a quality score, also commonly called a subjective Mean Opinion Score, MOS. The results can be used for evaluating the service, e.g. for finding improvements to make. However, such subjective panel tests are costly and time consuming.
Further problems include that panel tests and questionnaires can in practice only be conducted for a limited number of users which may not be representative for all users of a service, and that the feedback cannot be obtained in “real-time”, that is immediately after service delivery. Further problems include that considerable efforts must be spent to either conduct a panel test or distribute a questionnaire to a significant but still limited number of users and evaluate the results which could be more or less trustworthy.
Objective methods for estimating QoE have been developed as an alternative to subjective panel tests and questionnaires, which use a predefined “quality model” to automatically produce an opinion score by applying the quality model on technical parameters of a service delivery. However, the resulting opinion score may sometimes not be representative or truthful depending on the circumstances in real service deliveries which circumstances may not match or be valid for the used quality model.